Combined Ordered and Improved Trajectories for Large Scale Human Action Recognition

Recently, a video representation based on dense trajectories has been shown to outperform other human action recognition methods on several benchmark datasets. The trajectories capture the motion characteristics of different objects, for example human bodies, in spatial and temporal dimensions. In dense trajectories, points are sampled at uniform intervals in space and time and then tracked using a dense optical flow field over a fixed length time window of L frames (optimally 15) overlapping over the entire video. However, amongst these base trajectories, some continue for longer than duration L. These longer motion characteristics of objects may be more valuable than the information from the base trajectories or at least provide complementary information otherwise not captured. Thertefore, we propose a technique that searches for trajectories with longer duration and call these ‘ordered trajectories’. We apply these ordered trajectories in conjunction with the recent ‘improved trajectories’ (improved dense trajectories) approach on the UCF101 dataset. 1. Ordered Trajectories The overall layout of our proposed framework is shown in Fig. 1. Firstly, dense trajectories [4] are detected. The dense trajectories code available online1 [4] was used in all our experiments. The ‘ordered trajectories’ have been recently proposed by [2]. They were proposed to select those trajectories from, the base dense trajectories, that have a longer duration. Dense trajectories are usually computed over every overlapping sequence of L(= 15) frames. However, some trajectories can continue for varying periods beyond the fixed length of L. The ordered trajectories technique generates all such trajectories by matching the dense trajectories of every two consecutive frames. 1http://lear.inrialpes.fr/people/wang/dense_ trajectories Local descriptors – Motion Bound Histograms (MBH), Histograms of Oriented Gradients (HOG), Histogram of Optical Flow (HOF) – of the selected matching trajectories are accumulated, while the Trajectory Shape descriptor is computed from the generated ordered trajectories as follows. For a trajectory of given length L (number of frames) and containing a sequence of points Pt = (xt ,yt), the trajectory shape is described in terms of a sequence of displacement vectors ∆Pt = (Pt+1− pt) = (xt+1−xt ,yt+1−yt). The resulting vector is normalised by the sum of displacement vector magnitudes T = ∆Pt , ...,∆Pt+L−1 ∑t+L−1 j=t ||∆Pj|| (1) 2. Improved Trajectories Improved trajectories have been recently proposed and the code released by Wang et al. [5]. It is an improved version of the dense trajectories obtained by estimating the camera motion, which is estimated by matching feature points between frames using SURF descriptors and dense optical flow. The obtained matches are used to estimate a homography with RANSAC. Further, a human body detector is used to separate motion stemming from humans moving and from camera motion. The estimate is also used to cancel out possible camera motion from the optical flow. This technique has been shown to significantly improve motion-based descriptors such as MBH and HOF by removing apparent motion not related to the human body. In our experiments, we only use the camera motion compensated improved trajectories, without any human body detector. It is available online2 2http://lear.inrialpes.fr/people/wang/improved_